Pandas Profiling Report

Dataset statistics

Number of variables	29
Number of observations	100000
Missing cells	871235
Missing cells (%)	30.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	22.1 MiB
Average record size in memory	232.0 B

Variable types

CAT	20
NUM	7
BOOL	2

Warnings

`crash_date` has a high cardinality: 551 distinct values	High cardinality
`crash_time` has a high cardinality: 1440 distinct values	High cardinality
`location` has a high cardinality: 44605 distinct values	High cardinality
`on_street_name` has a high cardinality: 4327 distinct values	High cardinality
`off_street_name` has a high cardinality: 4897 distinct values	High cardinality
`cross_street_name` has a high cardinality: 22829 distinct values	High cardinality
`contributing_factor_vehicle_1` has a high cardinality: 54 distinct values	High cardinality
`vehicle_type_code1` has a high cardinality: 366 distinct values	High cardinality
`vehicle_type_code2` has a high cardinality: 385 distinct values	High cardinality
`vehicle_type_code_3` has a high cardinality: 64 distinct values	High cardinality
`longitude` is highly correlated with `latitude`	High correlation
`latitude` is highly correlated with `longitude`	High correlation
`number_of_motorist_injured` is highly correlated with `number_of_persons_injured`	High correlation
`number_of_persons_injured` is highly correlated with `number_of_motorist_injured`	High correlation
`borough` has 35026 (35.0%) missing values	Missing
`zip_code` has 35034 (35.0%) missing values	Missing
`latitude` has 8035 (8.0%) missing values	Missing
`longitude` has 8035 (8.0%) missing values	Missing
`location` has 8035 (8.0%) missing values	Missing
`on_street_name` has 26009 (26.0%) missing values	Missing
`off_street_name` has 52875 (52.9%) missing values	Missing
`cross_street_name` has 74033 (74.0%) missing values	Missing
`contributing_factor_vehicle_2` has 19243 (19.2%) missing values	Missing
`contributing_factor_vehicle_3` has 91239 (91.2%) missing values	Missing
`contributing_factor_vehicle_4` has 97760 (97.8%) missing values	Missing
`contributing_factor_vehicle_5` has 99333 (99.3%) missing values	Missing
`vehicle_type_code2` has 26589 (26.6%) missing values	Missing
`vehicle_type_code_3` has 91671 (91.7%) missing values	Missing
`vehicle_type_code_4` has 97853 (97.9%) missing values	Missing
`vehicle_type_code_5` has 99354 (99.4%) missing values	Missing
`latitude` is highly skewed (γ1 = -23.18863039)	Skewed
`cross_street_name` is uniformly distributed	Uniform
`collision_id` has unique values	Unique
`number_of_persons_injured` has 72699 (72.7%) zeros	Zeros
`number_of_pedestrians_injured` has 95454 (95.5%) zeros	Zeros
`number_of_motorist_injured` has 81887 (81.9%) zeros	Zeros

Reproduction

Analysis started	2020-12-11 09:15:39.024155
Analysis finished	2020-12-11 09:15:59.016567
Duration	19.99 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

collision_id
Real number (ℝ_≥0)

UNIQUE

Distinct	100000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4226109.341
Minimum	2568
Maximum	4353706
Zeros	0
Zeros (%)	0.0%
Memory size	781.2 KiB

Quantile statistics

Minimum	2568
5-th percentile	3665427.95
Q1	4182342.75
median	4300224
Q3	4328315.25
95-th percentile	4348345.05
Maximum	4353706
Range	4351138
Interquartile range (IQR)	145972.5

Descriptive statistics

Standard deviation	165356.0511
Coefficient of variation (CV)	0.03912725341
Kurtosis	45.22161792
Mean	4226109.341
Median Absolute Deviation (MAD)	51882.5
Skewness	-3.965406795
Sum	4.226109341e+11
Variance	2.734262364e+10
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
4196351	1	< 0.1%
4196884	1	< 0.1%
4303971	1	< 0.1%
4335798	1	< 0.1%
4159994	1	< 0.1%
4321918	1	< 0.1%
4167876	1	< 0.1%
4310832	1	< 0.1%
4318727	1	< 0.1%
4319172	1	< 0.1%
Other values (99990)	99990	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2568	1	< 0.1%
69010	1	< 0.1%
74294	1	< 0.1%
127733	1	< 0.1%
210591	1	< 0.1%

Value	Count	Frequency (%)
4353706	1	< 0.1%
4353705	1	< 0.1%
4353701	1	< 0.1%
4353672	1	< 0.1%
4353663	1	< 0.1%

crash_date
Categorical

HIGH CARDINALITY

Distinct	551
Distinct (%)	0.6%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

2019-07-19T00:00:00.000	664
2019-07-16T00:00:00.000	654
2019-07-26T00:00:00.000	648
2019-09-03T00:00:00.000	648
2019-07-29T00:00:00.000	646
Other values (546)	96740

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
2019-07-19T00:00:00.000	664	0.7%
2019-07-16T00:00:00.000	654	0.7%
2019-07-26T00:00:00.000	648	0.6%
2019-09-03T00:00:00.000	648	0.6%
2019-07-29T00:00:00.000	646	0.6%
2019-08-08T00:00:00.000	645	0.6%
2019-08-09T00:00:00.000	642	0.6%
2019-07-22T00:00:00.000	637	0.6%
2019-07-15T00:00:00.000	636	0.6%
2019-07-30T00:00:00.000	634	0.6%
Other values (541)	93546	93.5%

Frequencies of value counts

Unique

Unique	90 ?
Unique (%)	0.1%

Histogram of lengths of the category

Length

Max length	23
Median length	23
Mean length	23
Min length	23

crash_time
Categorical

HIGH CARDINALITY

Distinct	1440
Distinct (%)	1.4%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0:00	1637
17:00	1363
16:00	1360
14:00	1298
15:00	1246
Other values (1435)	93096

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
0:00	1637	1.6%
17:00	1363	1.4%
16:00	1360	1.4%
14:00	1298	1.3%
15:00	1246	1.2%
18:00	1231	1.2%
13:00	1153	1.2%
12:00	1103	1.1%
19:00	996	1.0%
10:00	971	1.0%
Other values (1430)	87642	87.6%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	5
Median length	5
Mean length	4.74399
Min length	4

borough
Categorical

MISSING

Distinct	5
Distinct (%)	< 0.1%
Missing	35026
Missing (%)	35.0%
Memory size	781.2 KiB

BROOKLYN	22118
QUEENS	18322
BRONX	11927
MANHATTAN	10637
STATEN ISLAND	1970

Frequencies
Length

Value	Count	Frequency (%)
BROOKLYN	22118	22.1%
QUEENS	18322	18.3%
BRONX	11927	11.9%
MANHATTAN	10637	10.6%
STATEN ISLAND	1970	2.0%
(Missing)	35026	35.0%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	13
Median length	6
Mean length	5.72932
Min length	3

zip_code
Real number (ℝ_≥0)

MISSING

Distinct	203
Distinct (%)	0.3%
Missing	35034
Missing (%)	35.0%
Infinite	0
Infinite (%)	0.0%

Mean	10901.65319
Minimum	10000
Maximum	11697
Zeros	0
Zeros (%)	0.0%
Memory size	781.2 KiB

Quantile statistics

Minimum	10000
5-th percentile	10013
Q1	10457
median	11209
Q3	11354
95-th percentile	11432
Maximum	11697
Range	1697
Interquartile range (IQR)	897

Descriptive statistics

Standard deviation	523.4949054
Coefficient of variation (CV)	0.04801977245
Kurtosis	-1.261718063
Mean	10901.65319
Median Absolute Deviation (MAD)	203
Skewness	-0.5985244273
Sum	708236801
Variance	274046.916
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
11207	1510	1.5%
11236	1175	1.2%
11212	1088	1.1%
11208	1071	1.1%
11385	1015	1.0%
11203	988	1.0%
11434	960	1.0%
11234	931	0.9%
11226	923	0.9%
11368	864	0.9%
Other values (193)	54441	54.4%
(Missing)	35034	35.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
10000	19	< 0.1%
10001	460	0.5%
10002	639	0.6%
10003	309	0.3%
10004	88	0.1%

Value	Count	Frequency (%)
11697	11	< 0.1%
11695	1	< 0.1%
11694	123	0.1%
11693	104	0.1%
11692	123	0.1%

latitude
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING
SKEWED

Distinct	33675
Distinct (%)	36.6%
Missing	8035
Missing (%)	8.0%
Infinite	0
Infinite (%)	0.0%

Mean	40.65191698
Minimum	0
Maximum	40.91217
Zeros	169
Zeros (%)	0.2%
Memory size	781.2 KiB

Quantile statistics

Minimum	0
5-th percentile	40.60009
Q1	40.667915
median	40.717724
Q3	40.785595
95-th percentile	40.864254
Maximum	40.91217
Range	40.91217
Interquartile range (IQR)	0.11768

Descriptive statistics

Standard deviation	1.746142914
Coefficient of variation (CV)	0.04295351963
Kurtosis	536.8860631
Mean	40.65191698
Median Absolute Deviation (MAD)	0.053046
Skewness	-23.18863039
Sum	3738553.545
Variance	3.049015076
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	169	0.2%
40.861862	79	0.1%
40.8047	56	0.1%
40.820305	52	0.1%
40.696033	48	< 0.1%
40.675735	48	< 0.1%
40.658577	47	< 0.1%
40.737785	45	< 0.1%
40.651863	43	< 0.1%
40.65965	42	< 0.1%
Other values (33665)	91336	91.3%
(Missing)	8035	8.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	169	0.2%
40.501465	1	< 0.1%
40.50331	1	< 0.1%
40.503387	1	< 0.1%
40.503414	1	< 0.1%

Value	Count	Frequency (%)
40.91217	1	< 0.1%
40.912117	1	< 0.1%
40.912018	1	< 0.1%
40.91038	1	< 0.1%
40.91032	2	< 0.1%

longitude
Real number (ℝ)

HIGH CORRELATION
MISSING

Distinct	26494
Distinct (%)	28.8%
Missing	8035
Missing (%)	8.0%
Infinite	0
Infinite (%)	0.0%

Mean	-73.78199499
Minimum	-201.23706
Maximum	0
Zeros	169
Zeros (%)	0.2%
Memory size	781.2 KiB

Quantile statistics

Minimum	-201.23706
5-th percentile	-74.015508
Q1	-73.96087
median	-73.91811
Q3	-73.86286
95-th percentile	-73.761
Maximum	0
Range	201.23706
Interquartile range (IQR)	0.09801

Descriptive statistics

Standard deviation	3.276307216
Coefficient of variation (CV)	-0.04440524028
Kurtosis	569.2955403
Mean	-73.78199499
Median Absolute Deviation (MAD)	0.04826
Skewness	18.42731855
Sum	-6785361.169
Variance	10.73418897
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	169	0.2%
-73.91282	83	0.1%
-73.89063	73	0.1%
-73.91243	61	0.1%
-73.89083	58	0.1%
-73.89686	53	0.1%
-73.98453	52	0.1%
-73.93755	47	< 0.1%
-73.96191	46	< 0.1%
-73.86536	46	< 0.1%
Other values (26484)	91277	91.3%
(Missing)	8035	8.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-201.23706	4	< 0.1%
-74.253006	1	< 0.1%
-74.250824	1	< 0.1%
-74.25076	1	< 0.1%
-74.25015	1	< 0.1%

Value	Count	Frequency (%)
0	169	0.2%
-73.700584	1	< 0.1%
-73.70073	1	< 0.1%
-73.70099	1	< 0.1%
-73.701004	1	< 0.1%

location
Categorical

HIGH CARDINALITY
MISSING

Distinct	44605
Distinct (%)	48.5%
Missing	8035
Missing (%)	8.0%
Memory size	781.2 KiB

(0.0, 0.0)	169
(40.861862, -73.91282)	79
(40.8047, -73.91243)	55
(40.820305, -73.89083)	52
(40.675735, -73.89686)	48
Other values (44600)	91562

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
(0.0, 0.0)	169	0.2%
(40.861862, -73.91282)	79	0.1%
(40.8047, -73.91243)	55	0.1%
(40.820305, -73.89083)	52	0.1%
(40.675735, -73.89686)	48	< 0.1%
(40.696033, -73.98453)	48	< 0.1%
(40.658577, -73.89063)	47	< 0.1%
(40.737785, -73.93496)	43	< 0.1%
(40.733536, -73.87035)	41	< 0.1%
(40.66496, -73.82226)	40	< 0.1%
Other values (44595)	91343	91.3%
(Missing)	8035	8.0%

Frequencies of value counts

Unique

Unique	29003 ?
Unique (%)	31.5%

Histogram of lengths of the category

Length

Max length	25
Median length	22
Mean length	20.20682
Min length	3

on_street_name
Categorical

HIGH CARDINALITY
MISSING

Distinct	4327
Distinct (%)	5.8%
Missing	26009
Missing (%)	26.0%
Memory size	781.2 KiB

BELT PARKWAY	1616
LONG ISLAND EXPRESSWAY	1053
BROOKLYN QUEENS EXPRESSWAY	956
BROADWAY	863
FDR DRIVE	852
Other values (4322)	68651

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
BELT PARKWAY	1616	1.6%
LONG ISLAND EXPRESSWAY	1053	1.1%
BROOKLYN QUEENS EXPRESSWAY	956	1.0%
BROADWAY	863	0.9%
FDR DRIVE	852	0.9%
GRAND CENTRAL PKWY	820	0.8%
ATLANTIC AVENUE	717	0.7%
MAJOR DEEGAN EXPRESSWAY	674	0.7%
CROSS BRONX EXPY	652	0.7%
CROSS ISLAND PARKWAY	605	0.6%
Other values (4317)	65183	65.2%
(Missing)	26009	26.0%

Frequencies of value counts

Unique

Unique	1369 ?
Unique (%)	1.9%

Histogram of lengths of the category

Length

Max length	32
Median length	32
Mean length	24.45739
Min length	3

off_street_name
Categorical

HIGH CARDINALITY
MISSING

Distinct	4897
Distinct (%)	10.4%
Missing	52875
Missing (%)	52.9%
Memory size	781.2 KiB

3 AVENUE	432
BROADWAY	424
2 AVENUE	340
LINDEN BOULEVARD	280
5 AVENUE	247
Other values (4892)	45402

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
3 AVENUE	432	0.4%
BROADWAY	424	0.4%
2 AVENUE	340	0.3%
LINDEN BOULEVARD	280	0.3%
5 AVENUE	247	0.2%
ATLANTIC AVENUE	240	0.2%
1 AVENUE	237	0.2%
7 AVENUE	229	0.2%
PARK AVENUE	222	0.2%
QUEENS BOULEVARD	218	0.2%
Other values (4887)	44256	44.3%
(Missing)	52875	52.9%

Frequencies of value counts

Unique

Unique	1704 ?
Unique (%)	3.6%

Histogram of lengths of the category

Length

Max length	32
Median length	3
Mean length	7.80378
Min length	1

cross_street_name
Categorical

HIGH CARDINALITY
MISSING
UNIFORM

Distinct	22829
Distinct (%)	87.9%
Missing	74033
Missing (%)	74.0%
Memory size	781.2 KiB

772 EDGEWATER ROAD	35
501 GATEWAY DRIVE	21
90-15 QUEENS BOULEVARD	19
123-01 ROOSEVELT AVENUE	18
2100 BARTOW AVENUE	14
Other values (22824)	25860

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
772 EDGEWATER ROAD	35	< 0.1%
501 GATEWAY DRIVE	21	< 0.1%
90-15 QUEENS BOULEVARD	19	< 0.1%
123-01 ROOSEVELT AVENUE	18	< 0.1%
2100 BARTOW AVENUE	14	< 0.1%
985 RICHMOND AVENUE	13	< 0.1%
815 HUTCHINSON RIVER PARKWAY	12	< 0.1%
1 ORCHARD BEACH ROAD	12	< 0.1%
355 FOOD CENTER DRIVE	12	< 0.1%
135-05 20 AVENUE	12	< 0.1%
Other values (22819)	25799	25.8%
(Missing)	74033	74.0%

Frequencies of value counts

Unique

Unique	20794 ?
Unique (%)	80.1%

Histogram of lengths of the category

Length

Max length	40
Median length	3
Mean length	12.60779
Min length	3

number_of_persons_injured
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	13
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.37196
Minimum	0
Maximum	15
Zeros	72699
Zeros (%)	72.7%
Memory size	781.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	1
95-th percentile	2
Maximum	15
Range	15
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	0.7439161865
Coefficient of variation (CV)	1.999989748
Kurtosis	16.5808926
Mean	0.37196
Median Absolute Deviation (MAD)	0
Skewness	3.147118256
Sum	37196
Variance	0.5534112925
Monotocity	Not monotonic

Histogram with fixed size bins (bins=13)

Value	Count	Frequency (%)
0	72699	72.7%
1	21011	21.0%
2	4125	4.1%
3	1308	1.3%
4	523	0.5%
5	196	0.2%
6	77	0.1%
7	36	< 0.1%
8	14	< 0.1%
9	5	< 0.1%
Other values (3)	6	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	72699	72.7%
1	21011	21.0%
2	4125	4.1%
3	1308	1.3%
4	523	0.5%

Value	Count	Frequency (%)
15	1	< 0.1%
11	3	< 0.1%
10	2	< 0.1%
9	5	< 0.1%
8	14	< 0.1%

number_of_persons_killed
Categorical

Distinct	4
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0	99816
1	176
2	7
3	1

Frequencies
Length

Value	Count	Frequency (%)
0	99816	99.8%
1	176	0.2%
2	7	< 0.1%
3	1	< 0.1%

Frequencies of value counts

Unique

Unique	1 ?
Unique (%)	< 0.1%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

number_of_pedestrians_injured
Real number (ℝ_≥0)

ZEROS

Distinct	7
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.04739
Minimum	0
Maximum	6
Zeros	95454
Zeros (%)	95.5%
Memory size	781.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	0
Maximum	6
Range	6
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	0.2234383296
Coefficient of variation (CV)	4.714883512
Kurtosis	38.41899762
Mean	0.04739
Median Absolute Deviation (MAD)	0
Skewness	5.270026474
Sum	4739
Variance	0.04992468715
Monotocity	Not monotonic

Histogram with fixed size bins (bins=7)

Value	Count	Frequency (%)
0	95454	95.5%
1	4383	4.4%
2	142	0.1%
3	17	< 0.1%
6	2	< 0.1%
5	1	< 0.1%
4	1	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	95454	95.5%
1	4383	4.4%
2	142	0.1%
3	17	< 0.1%
4	1	< 0.1%

Value	Count	Frequency (%)
6	2	< 0.1%
5	1	< 0.1%
4	1	< 0.1%
3	17	< 0.1%
2	142	0.1%

number_of_pedestrians_killed
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0	99936
1	64

Common Values
Chart

Value	Count	Frequency (%)
0	99936	99.9%
1	64	0.1%

number_of_cyclist_injured
Categorical

Distinct	4
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0	95147
1	4744
2	107
3	2

Frequencies
Length

Value	Count	Frequency (%)
0	95147	95.1%
1	4744	4.7%
2	107	0.1%
3	2	< 0.1%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

number_of_cyclist_killed
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0	99975
1	25

Common Values
Chart

Value	Count	Frequency (%)
0	99975	> 99.9%
1	25	< 0.1%

number_of_motorist_injured
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	13
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.27492
Minimum	0
Maximum	15
Zeros	81887
Zeros (%)	81.9%
Memory size	781.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0
95-th percentile	2
Maximum	15
Range	15
Interquartile range (IQR)	0

Descriptive statistics

Standard deviation	0.711058401
Coefficient of variation (CV)	2.586419326
Kurtosis	21.76181987
Mean	0.27492
Median Absolute Deviation (MAD)	0
Skewness	3.819224309
Sum	27492
Variance	0.5056040496
Monotocity	Not monotonic

Histogram with fixed size bins (bins=13)

Value	Count	Frequency (%)
0	81887	81.9%
1	12243	12.2%
2	3767	3.8%
3	1259	1.3%
4	523	0.5%
5	189	0.2%
6	73	0.1%
7	34	< 0.1%
8	14	< 0.1%
9	5	< 0.1%
Other values (3)	6	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	81887	81.9%
1	12243	12.2%
2	3767	3.8%
3	1259	1.3%
4	523	0.5%

Value	Count	Frequency (%)
15	1	< 0.1%
11	3	< 0.1%
10	2	< 0.1%
9	5	< 0.1%
8	14	< 0.1%

number_of_motorist_killed
Categorical

Distinct	4
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	781.2 KiB

0	99904
1	89
2	6
3	1

Frequencies
Length

Value	Count	Frequency (%)
0	99904	99.9%
1	89	0.1%
2	6	< 0.1%
3	1	< 0.1%

Frequencies of value counts

Unique

Unique	1 ?
Unique (%)	< 0.1%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

contributing_factor_vehicle_1
Categorical

HIGH CARDINALITY

Distinct	54
Distinct (%)	0.1%
Missing	371
Missing (%)	0.4%
Memory size	781.2 KiB

Driver Inattention/Distraction	25605
Unspecified	25253
Following Too Closely	7530
Failure to Yield Right-of-Way	6023
Backing Unsafely	4033
Other values (49)	31185

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Driver Inattention/Distraction	25605	25.6%
Unspecified	25253	25.3%
Following Too Closely	7530	7.5%
Failure to Yield Right-of-Way	6023	6.0%
Backing Unsafely	4033	4.0%
Passing or Lane Usage Improper	3979	4.0%
Passing Too Closely	3676	3.7%
Other Vehicular	3071	3.1%
Unsafe Lane Changing	2588	2.6%
Unsafe Speed	2447	2.4%
Other values (44)	15424	15.4%

Frequencies of value counts

Unique

Unique	2 ?
Unique (%)	< 0.1%

Histogram of lengths of the category

Length

Max length	53
Median length	21
Mean length	21.15973
Min length	3

contributing_factor_vehicle_2
Categorical

MISSING

Distinct	47
Distinct (%)	0.1%
Missing	19243
Missing (%)	19.2%
Memory size	781.2 KiB

Unspecified	67739
Driver Inattention/Distraction	5284
Following Too Closely	1296
Other Vehicular	1249
Passing or Lane Usage Improper	802
Other values (42)	4387

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Unspecified	67739	67.7%
Driver Inattention/Distraction	5284	5.3%
Following Too Closely	1296	1.3%
Other Vehicular	1249	1.2%
Passing or Lane Usage Improper	802	0.8%
Failure to Yield Right-of-Way	716	0.7%
Passing Too Closely	538	0.5%
Unsafe Lane Changing	402	0.4%
Unsafe Speed	383	0.4%
Traffic Control Disregarded	370	0.4%
Other values (37)	1978	2.0%
(Missing)	19243	19.2%

Frequencies of value counts

Unique

Unique	2 ?
Unique (%)	< 0.1%

Histogram of lengths of the category

Length

Max length	53
Median length	11
Mean length	11.33302
Min length	3

contributing_factor_vehicle_3
Categorical

MISSING

Distinct	30
Distinct (%)	0.3%
Missing	91239
Missing (%)	91.2%
Memory size	781.2 KiB

Unspecified	8197
Following Too Closely	176
Other Vehicular	171
Driver Inattention/Distraction	118
Reaction to Uninvolved Vehicle	16
Other values (25)	83

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Unspecified	8197	8.2%
Following Too Closely	176	0.2%
Other Vehicular	171	0.2%
Driver Inattention/Distraction	118	0.1%
Reaction to Uninvolved Vehicle	16	< 0.1%
Unsafe Speed	15	< 0.1%
Pavement Slippery	12	< 0.1%
Driver Inexperience	5	< 0.1%
Passing or Lane Usage Improper	5	< 0.1%
Driverless/Runaway Vehicle	4	< 0.1%
Other values (20)	42	< 0.1%
(Missing)	91239	91.2%

Frequencies of value counts

Unique

Unique	7 ?
Unique (%)	0.1%

Histogram of lengths of the category

Length

Max length	53
Median length	3
Mean length	3.75874
Min length	3

contributing_factor_vehicle_4
Categorical

MISSING

Distinct	12
Distinct (%)	0.5%
Missing	97760
Missing (%)	97.8%
Memory size	781.2 KiB

Unspecified	2107
Other Vehicular	54
Following Too Closely	41
Driver Inattention/Distraction	22
Pavement Slippery	4
Other values (7)	12

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Unspecified	2107	2.1%
Other Vehicular	54	0.1%
Following Too Closely	41	< 0.1%
Driver Inattention/Distraction	22	< 0.1%
Pavement Slippery	4	< 0.1%
Reaction to Uninvolved Vehicle	3	< 0.1%
Unsafe Speed	3	< 0.1%
Aggressive Driving/Road Rage	2	< 0.1%
Obstruction/Debris	1	< 0.1%
Alcohol Involvement	1	< 0.1%
Other values (2)	2	< 0.1%
(Missing)	97760	97.8%

Frequencies of value counts

Unique

Unique	4 ?
Unique (%)	0.2%

Histogram of lengths of the category

Length

Max length	30
Median length	3
Mean length	3.19114
Min length	3

contributing_factor_vehicle_5
Categorical

MISSING

Distinct	9
Distinct (%)	1.3%
Missing	99333
Missing (%)	99.3%
Memory size	781.2 KiB

Unspecified	622
Other Vehicular	24
Following Too Closely	12
Driver Inattention/Distraction	3
Pavement Slippery	2
Other values (4)	4

Frequencies
Length

Value	Count	Frequency (%)
Unspecified	622	0.6%
Other Vehicular	24	< 0.1%
Following Too Closely	12	< 0.1%
Driver Inattention/Distraction	3	< 0.1%
Pavement Slippery	2	< 0.1%
Passing Too Closely	1	< 0.1%
Reaction to Uninvolved Vehicle	1	< 0.1%
Unsafe Speed	1	< 0.1%
Obstruction/Debris	1	< 0.1%
(Missing)	99333	99.3%

Frequencies of value counts

Unique

Unique	4 ?
Unique (%)	0.6%

Histogram of lengths of the category

Length

Max length	30
Median length	3
Mean length	3.05656
Min length	3

vehicle_type_code1
Categorical

HIGH CARDINALITY

Distinct	366
Distinct (%)	0.4%
Missing	740
Missing (%)	0.7%
Memory size	781.2 KiB

Sedan	46790
Station Wagon/Sport Utility Vehicle	35766
Taxi	3478
Pick-up Truck	2615
Box Truck	1946
Other values (361)	8665

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Sedan	46790	46.8%
Station Wagon/Sport Utility Vehicle	35766	35.8%
Taxi	3478	3.5%
Pick-up Truck	2615	2.6%
Box Truck	1946	1.9%
Bike	1437	1.4%
Bus	1125	1.1%
Motorcycle	921	0.9%
Tractor Truck Diesel	751	0.8%
Van	572	0.6%
Other values (356)	3859	3.9%
(Missing)	740	0.7%

Frequencies of value counts

Unique

Unique	225 ?
Unique (%)	0.2%

Histogram of lengths of the category

Length

Max length	38
Median length	5
Mean length	16.23289
Min length	1

vehicle_type_code2
Categorical

HIGH CARDINALITY
MISSING

Distinct	385
Distinct (%)	0.5%
Missing	26589
Missing (%)	26.6%
Memory size	781.2 KiB

Sedan	31369
Station Wagon/Sport Utility Vehicle	24773
Bike	3586
Taxi	2300
Pick-up Truck	2282
Other values (380)	9101

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Sedan	31369	31.4%
Station Wagon/Sport Utility Vehicle	24773	24.8%
Bike	3586	3.6%
Taxi	2300	2.3%
Pick-up Truck	2282	2.3%
Box Truck	2146	2.1%
Bus	1011	1.0%
Tractor Truck Diesel	763	0.8%
Motorcycle	731	0.7%
Van	537	0.5%
Other values (375)	3913	3.9%
(Missing)	26589	26.6%

Frequencies of value counts

Unique

Unique	218 ?
Unique (%)	0.3%

Histogram of lengths of the category

Length

Max length	38
Median length	5
Mean length	12.37042
Min length	2

vehicle_type_code_3
Categorical

HIGH CARDINALITY
MISSING

Distinct	64
Distinct (%)	0.8%
Missing	91671
Missing (%)	91.7%
Memory size	781.2 KiB

Sedan	4129
Station Wagon/Sport Utility Vehicle	3380
Pick-up Truck	195
Taxi	187
Box Truck	73
Other values (59)	365

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Sedan	4129	4.1%
Station Wagon/Sport Utility Vehicle	3380	3.4%
Pick-up Truck	195	0.2%
Taxi	187	0.2%
Box Truck	73	0.1%
Motorcycle	46	< 0.1%
Bike	46	< 0.1%
Bus	41	< 0.1%
Van	40	< 0.1%
Tractor Truck Diesel	37	< 0.1%
Other values (54)	155	0.2%
(Missing)	91671	91.7%

Frequencies of value counts

Unique

Unique	32 ?
Unique (%)	0.4%

Histogram of lengths of the category

Length

Max length	35
Median length	3
Mean length	4.20942
Min length	2

vehicle_type_code_4
Categorical

MISSING

Distinct	31
Distinct (%)	1.4%
Missing	97853
Missing (%)	97.9%
Memory size	781.2 KiB

Sedan	1143
Station Wagon/Sport Utility Vehicle	847
Pick-up Truck	49
Taxi	32
Box Truck	12
Other values (26)	64

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Sedan	1143	1.1%
Station Wagon/Sport Utility Vehicle	847	0.8%
Pick-up Truck	49	< 0.1%
Taxi	32	< 0.1%
Box Truck	12	< 0.1%
Convertible	10	< 0.1%
Bus	10	< 0.1%
Motorcycle	7	< 0.1%
Dump	5	< 0.1%
Tractor Truck Diesel	4	< 0.1%
Other values (21)	28	< 0.1%
(Missing)	97853	97.9%

Frequencies of value counts

Unique

Unique	16 ?
Unique (%)	0.7%

Histogram of lengths of the category

Length

Max length	35
Median length	3
Mean length	3.30334
Min length	3

vehicle_type_code_5
Categorical

MISSING

Distinct	18
Distinct (%)	2.8%
Missing	99354
Missing (%)	99.4%
Memory size	781.2 KiB

Sedan	326
Station Wagon/Sport Utility Vehicle	263
Pick-up Truck	23
Taxi	6
Box Truck	4
Other values (13)	24

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Sedan	326	0.3%
Station Wagon/Sport Utility Vehicle	263	0.3%
Pick-up Truck	23	< 0.1%
Taxi	6	< 0.1%
Box Truck	4	< 0.1%
Van	4	< 0.1%
Motorcycle	4	< 0.1%
Convertible	3	< 0.1%
PK	3	< 0.1%
Bus	2	< 0.1%
Other values (8)	8	< 0.1%
(Missing)	99354	99.4%

Frequencies of value counts

Unique

Unique	8 ?
Unique (%)	1.2%

Histogram of lengths of the category

Length

Max length	35
Median length	3
Mean length	3.0943
Min length	2

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

First rows

	collision_id	crash_date	crash_time	borough	zip_code	latitude	longitude	location	on_street_name	off_street_name	cross_street_name	number_of_persons_injured	number_of_pedestrians_injured	number_of_motorist_injured	contributing_factor_vehicle_1	contributing_factor_vehicle_2	contributing_factor_vehicle_3	contributing_factor_vehicle_4	contributing_factor_vehicle_5	vehicle_type_code1	vehicle_type_code2	vehicle_type_code_3	vehicle_type_code_4	vehicle_type_code_5
0	3654181	2017-04-18T00:00:00.000	23:10	STATEN ISLAND	10312.0	40.536728	-74.193344	(40.536728, -74.193344)	NaN	NaN	243 DARLINGTON AVENUE	0	0	0	Driver Inattention/Distraction	Unspecified	NaN	NaN	NaN	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN	NaN
1	3665311	2017-05-06T00:00:00.000	13:00	BRONX	10472.0	40.829052	-73.850380	(40.829052, -73.85038)	CASTLE HILL AVENUE	BLACKROCK AVENUE	NaN	1	1	0	Failure to Yield Right-of-Way	NaN	NaN	NaN	NaN	Sedan	NaN	NaN	NaN	NaN
2	3658491	2017-04-27T00:00:00.000	17:15	QUEENS	11420.0	40.677303	-73.804565	(40.677303, -73.804565)	135 STREET	FOCH BOULEVARD	NaN	0	0	0	Driver Inattention/Distraction	Unspecified	NaN	NaN	NaN	Sedan	Sedan	NaN	NaN	NaN
3	3666554	2017-05-09T00:00:00.000	20:10	NaN	NaN	40.624958	-74.145775	(40.624958, -74.145775)	FOREST AVENUE	RICHMOND AVENUE	NaN	1	0	1	Unspecified	Unspecified	Unspecified	NaN	NaN	Motorcycle	Sedan	Bus	NaN	NaN
4	3653269	2017-04-18T00:00:00.000	14:00	BRONX	10456.0	40.828846	-73.903120	(40.828846, -73.90312)	NaN	NaN	1167 BOSTON ROAD	0	0	0	Driver Inattention/Distraction	Unspecified	NaN	NaN	NaN	Sedan	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
5	3666365	2017-05-08T00:00:00.000	10:33	NaN	NaN	40.556454	-74.207770	(40.556454, -74.20777)	WEST SHORE EXPRESSWAY	NaN	NaN	0	0	0	Unsafe Lane Changing	Unspecified	NaN	NaN	NaN	Sedan	Sedan	NaN	NaN	NaN
6	3666842	2017-05-10T00:00:00.000	6:10	NaN	NaN	40.740025	-73.976260	(40.740025, -73.97626)	1 AVENUE	EAST 28 STREET	NaN	0	0	0	Passing or Lane Usage Improper	Unspecified	NaN	NaN	NaN	Taxi	Box Truck	NaN	NaN	NaN
7	3657123	2017-04-24T00:00:00.000	9:30	BROOKLYN	11203.0	40.651646	-73.932330	(40.651646, -73.93233)	EAST 48 STREET	CHURCH AVENUE	NaN	0	0	0	Other Vehicular	Other Vehicular	NaN	NaN	NaN	Station Wagon/Sport Utility Vehicle	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
8	3651039	2017-04-14T00:00:00.000	13:00	NaN	NaN	40.751800	-73.817314	(40.7518, -73.817314)	ROBINSON STREET	NaN	NaN	0	0	0	Passing Too Closely	Unspecified	NaN	NaN	NaN	Sedan	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
9	3661896	2017-05-02T00:00:00.000	1:00	BRONX	10474.0	40.816864	-73.882744	(40.816864, -73.882744)	NaN	NaN	772 EDGEWATER ROAD	0	0	0	Unspecified	NaN	NaN	NaN	NaN	Pick-up Truck	NaN	NaN	NaN	NaN

Last rows

	collision_id	crash_date	crash_time	borough	zip_code	latitude	longitude	location	on_street_name	off_street_name	cross_street_name	contributing_factor_vehicle_1	contributing_factor_vehicle_2	contributing_factor_vehicle_3	contributing_factor_vehicle_4	contributing_factor_vehicle_5	vehicle_type_code1	vehicle_type_code2	vehicle_type_code_3	vehicle_type_code_4	vehicle_type_code_5
99990	4238828	2019-11-08T00:00:00.000	19:20	BROOKLYN	11218.0	NaN	NaN	NaN	OCEAN PARKWAY	AVENUE C	NaN	Unspecified	Unspecified	NaN	NaN	NaN	Sedan	Sedan	NaN	NaN	NaN
99991	4239244	2019-11-11T00:00:00.000	15:55	NaN	NaN	40.661540	-73.982740	(40.66154, -73.98274)	16 STREET	NaN	NaN	Outside Car Distraction	Unspecified	NaN	NaN	NaN	Sedan	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
99992	4240499	2019-11-13T00:00:00.000	11:00	BRONX	10461.0	40.836597	-73.840546	(40.836597, -73.840546)	NaN	NaN	1332 COMMERCE AVENUE	Unspecified	Unspecified	NaN	NaN	NaN	Pick-up Truck	Sedan	NaN	NaN	NaN
99993	4252028	2019-12-04T00:00:00.000	7:00	QUEENS	11385.0	40.703407	-73.883484	(40.703407, -73.883484)	NaN	NaN	71-17 69 STREET	Passing Too Closely	Unspecified	NaN	NaN	NaN	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN	NaN
99994	4242657	2019-11-15T00:00:00.000	13:05	BROOKLYN	11206.0	40.701862	-73.943830	(40.701862, -73.94383)	WHIPPLE STREET	BROADWAY	NaN	Following Too Closely	Unspecified	NaN	NaN	NaN	Sedan	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
99995	4244961	2019-11-20T00:00:00.000	15:00	BROOKLYN	11210.0	40.618893	-73.946420	(40.618893, -73.94642)	NaN	NaN	1314 EAST 29 STREET	Unspecified	NaN	NaN	NaN	NaN	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN	NaN
99996	4250093	2019-12-01T00:00:00.000	11:22	QUEENS	11367.0	40.723380	-73.814750	(40.72338, -73.81475)	NaN	NaN	150-62 76 ROAD	Unspecified	Unspecified	NaN	NaN	NaN	Station Wagon/Sport Utility Vehicle	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN
99997	4245290	2019-11-21T00:00:00.000	21:30	BROOKLYN	11249.0	40.710820	-73.968530	(40.71082, -73.96853)	BROADWAY	KENT AVENUE	NaN	Passing Too Closely	Unspecified	NaN	NaN	NaN	Sedan	Box Truck	NaN	NaN	NaN
99998	4243646	2019-11-18T00:00:00.000	17:28	BROOKLYN	11234.0	40.631180	-73.928185	(40.63118, -73.928185)	NaN	NaN	1695 UTICA AVENUE	Driver Inattention/Distraction	Unspecified	NaN	NaN	NaN	Sedan	Bus	NaN	NaN	NaN
99999	4247517	2019-11-17T00:00:00.000	20:42	MANHATTAN	10017.0	40.750760	-73.968430	(40.75076, -73.96843)	EAST 45 STREET	1 AVENUE	NaN	Driver Inattention/Distraction	Driver Inattention/Distraction	NaN	NaN	NaN	Sedan	Station Wagon/Sport Utility Vehicle	NaN	NaN	NaN

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows